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Overcoming obstacles 

Analytical Pitfalls and Stumbling Blocks 
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Soothsayers used to examine the entrails of animals 
to predict the future. Today, we intelligence anaylsts 
rely on far more sophisticated sources to underpin 
our appraisals of what lies ahead. Nevertheless, we 
often do not appear to obtain much better results that 
our more primitive predecessors. 

On some occasions our seeming inability to figure 
out what comes next is caused by the way we ap- 
proach our work, rather than a failure of intellect or 
imagination. Moreover, there are times when we ac- 
tually get it right but somehow are unable to per- 
suade our superiors or the policymakers of the ac- 
curacy of our judgments. 

The following is a list and disucssion of what I con- 
tend are several barriers to good intelligence analy- 
sis. Most examples relate to life in CIA's Directorate 
of Intelligence (DI). I believe that they also are 
generally applicable to intelligence producers in the 
community. I present this to alert junior people to 
potential problems and to remind seniors that there 
are traps for even the most experienced. 

Defensive Analysis 

Those analysts who are brave enough to make judg- 
ments usually make far more good ones than bad. 
The best and brightest analysts that I have known 
have never shirked from making a judgment, and 
promotions go more often to analytic risk-takers. If 
one wants to join their ranks, the following pitfalls 
should be avoided: 

• Echoing the opinions of collectors in the field, 
who usually are not privy to all-source intelli- 
gence. 

• Being comfortable as part of a consensus. You can 
have a lot of company in making a wrong judg- 
ment. 



• Defining prediction so narrowly that you really do 
not do it at all. 

• Setting up a series of distant hypotheses to avoid 
a short-term prediction. For example, an assess- 
ment of how a united Korea would likely deal 
with the US probably would not be helpful if it 
did not offer insight on when and how unification 
would be likely to occur. 

• Establishing a straight-line projection in lieu of 
real analysis. 

The Use of History 

A look at a nation's or region's history is often the 
logical starting point for an intelligence officer. For 
too many, however, it also seems to be the stopping 
point, particularly in terms of analytical history. The 
fact that something occurred a certain way 10 or 20 
years ago can easily become the basis for a judgment 
that the same thing is happening today. While such 
an approach often provides the correct answer, it 
almost guarantees failure in predicting new develop- 
ments. In using history, always look for what is new. 
Are there real parallels, or are today's events being 
forced into yesterday's mold? 

Applying Labels 

To simplify presentation, we type individuals and is- 
sues. Everyone is put into his or her group — 
conservatives and liberals, reactionaries and radicals, 
hardliners and softliners. Too often the reader is ex- 
pected to understand precisely what the terms mean. 
If there is a chance that your reader might be mis- 
lead, add a short definition. 
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Underestimating Situations 

For many years we put considerable stock in the per- 
manence of repression. The cant was that "the re- 
gime's pervasive and efficient secret police will be 
able to handle dissent." We also invented the "long- 
suffering peasant" who would put up forever with a 
situation that would send Americans to the barri- 
cades. Finally, we seemingly closed the loop with the 
judgment that, in the absence of evidence of large- 
scale opposition to a system, the people had to have 
bought what their governments were selling. But 
from time to time, and especially during the past few 
years, intelligence analysts have been brought up 
short when highly controlled regimes rapidly fell 
apart. What did we miss? 

I would argue that the myth we created of the degree 
of political control and the impact of societal inertia 
has been a major barrier to our understanding of 
change and our ability to recognize early the signs of 
an impending national or regional explosion. Once 
we conclude that a government is essentially un- 
challenged, virtually all resources and analytic effort 
can be focused on military and technological de- 
velopments, leadership shifts, foreign policy adven- 
tures, and economic problems. 

If we want to be relevant, bread-and-butter issues 
have to be at the top of our list. I do believe, 
however, that we have to pay more attention to un- 
derstanding societal change. 

This process often proceeds at a slow pace. To meas- 
ure it, we identify and weigh the importance of small 
breaks with the past and evaluate how such change 
will affect a government's ability to survive. 

Examples abound. The children of the revolution be- 
gin to replace their parents. Ideology is ridiculed. 
People begin to focus on the disparity between the 
haves and the have nots. Corruption erodes the lead- 
er's authority. Economic ineptness and bureaucratic 
bungling foster anger, resentment and frustration. 
Foreign wars fuel national rage as casualties mount. 
Young people reject the values of their own society 
for those of another. Although we have not ignored 
such changes, we have not concentrated on develop- 
ing the analytic tools to the extent that they can help 
us do a better job in anticipating landmark events. 



In looking at societal change, an analyst probably 
should not dwell too long on the differences between 
"them and us," to put much stock in "national 
character." People are likely to have the same needs, 
goals and desires wherever they live. Even though 
people adapt to the mores of their particular societies 
in order to survive, it does not mean that they have 
to like them. If logic suggests that people should be 
unhappy, they probably are. And at some point they 
will be likely to try to change their situation. 

Publish or Perish 

Intelligence officers use many different vehicles to 
transmit their messages. Many often appear to con- 
sider that the daily current intelligence publications 
have the most impact because the President, cabinet 
members, and other senior policymakers read them. 
Nevertheless, I see two serious problems stemming 
from the way we produce current intelligence — 
publishing too fast and too often. 

In my view, many analysts and managers believe that 
the primary goal is to scoop the commercial compe- 
tition. This approach sometimes means that ideas are 
vetted prematurely, before full analytical resources 
can be applied. As electronic dissemination is in- 
creasingly used, analysts will almost certainly come 
under even greater pressure to get a story out. 

Some analysts have said that they also have been 
pushed to publish because they are the recipients of 
expensive-to-obtain technical intelligence. Using the 
material in a product supposedly justifies the cost of 
collecting it, even though it adds little to the body of 
knowledge. 



Getting Locked In 

On really "hot" topics, the policymakers invariably 
will want a continuing flow of information. The lack 
of new information or insights, however, can lead to 
the repetition of analysis, with the consequence that 
a judgment too often stated becomes difficult to 
change. 
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This problem is compounded by a demand for con- 
tinuity and consistency. Most analysts have been ad- 
vised from time to time that it is bad form to keep 
taking their consumers up one hill and then down the 
next. There is, at least, an implicit command to stick 
to initial judgments until they are contradicted by 
strong evidence. From a public relations standpoint, 
this probably is the right thing to do. But this posi- 
tion would seem to impede the prompt transmittal of 
reports or messages to policymakers, thereby limiting 
their available options in dealing with the problems 
at hand. 

At a minimum, analysts should do a periodic zero- 
based review of their key judgments to ensure that 
pressures to write are not interfering with their abil- 
ity to determine Jhe facts of a given situation. 

Policy Relevance 

In the Agency's early years, managers and analysts 
seemingly adopted the view that knowledge was 
sought for knowledge's sake. This led to a debate be- 
tween those who believed that remaining aloof from 
policymakers was the only way to keep their analyti- 
cal purity and those who believed that purity without 
relevance made no sense. The latter group won the 
argument. 

Analysts and managers are now expected to establish 
effective contact with policymakers, and we have 
largely succeeded in plugging into the policy 
process. As a result of our efforts to march in step 
with policymakers, however, we appear less inclined 
to produce premonitory intelligence. This perception 
is based on a sampling of intelligence publications 
over time. I have gained the clear impression that 
most papers that I read now seem to emphasize what 
is happening and why, or what happened and why. In 
the interest of policy relevance we appear to be 
training a generation of intelligence newspersons and 
historians. 



Overcoming Bias 

A fair amount of time, energy, and emotion recently 
has been devoted to the debate on "politicization." 
There are a few points around the periphery of the 
discussion that are worth noting. 



First, managers in the DI are likely to draw similar 
conclusions from the same overt sources of informa- 
tion as policymakers. I would further argue that un- 
der most circumstances intelligence managers proba- 
bly are not captives of their consumers; they are 
likely to share the same view of the world, and they 
probably would be as skeptical as their clients if 
someone challenges what they see as reality. 

Assuming these thoughts are generally true, an 
analyst meeting resistance from a supervisor to a 
change in judgment and seeking a higher probability 
of getting his or her new or different interpretation 
into print should initially assume that the manager 
does not find the new evidence or analysis compel- 
ling. If there is good will on the part of the reviewer, 
the analyst has a reasonable chance at overcoming 
the former's doubts and reservations. But if the 
manager's resistance is politically motivated — a far 
lower probability in my mind — then nothing will 
move him or her. Consequently, an analyst should 
envision a situation that will permit success. 

Second, we focus on mangers as politicizers, and 
often forget that analysts can also have agendas other 
than producing the best unbiased analysis. There 
have been times in the past when one could question 
whether a particular line of analysis was aimed at 
enlightening or directing US policy. Even the way 
analysts package their judgments can lead to uncer- 
tainty about motivation. For example, I recall the 
plaintive remark of a branch chief: "I want to tell 
people the truth, but do we have to rub their noses in 
it?" 

As to inadvertent bias in analysis, there are numer- 
ous traps. On many occasions we DI military 
analysts expressed delight that our views placed us 
dead center between the Defense Department ("on 
the right") and the State Department ("on the left"). 
That may have been the best place to be, but in 
20-20 hindsight, one wonders how much of our anal- 
ysis was influenced by our desire not to adopt the 
positions of our competitors. 

While politicization can be a major impediment to 
providing the best intelligence, do not focus on it to 
the exclusion of equally important barriers. The key 
is to understand the nature of the problem. 



1 5 Confifrarrtial 

_Approved for Release: 2014/07/29 C00622865 



Approved for Release: 2014/07/29 C00622865 

Confidential 



-(b)(3)(H)- 



Blocks 



Writing to Communicate 

One of the rites of passage for most new DI analysts 
is a briefing on the DI writing style. The ostensible 
purpose is to prepare people for formatting their 
products, but the end result frequently is to intimi- 
date the new employee. The following is a composite 
of some of the briefings I have overheard: 

"Good morning, Mr. Phelps. Your mission, if 
you choose to accept it, is to learn to write in 
the DI style. This style is unique in the world; 
we believe that it came to the original intelli- 
gence analyst in a vision after he fasted for 40 
days in the desert. We put our judgments up 
front, use topic sentences, keep all sentences 
short and to the point, avoid exotic words, and 
put an implications section at the back of the 
paper to transmit the full impact of the mes- 
sage. It will probably take you years to learn it, 
and the odds are that you will never succeed. If 
you fail, your branch chief will disavow ever 
knowing you." 

I offer an alternative speech: 

"Good morning, Mr. Phelps. I have a mission 
for you that is far from impossible. I want you 
to take those skills that you have been using all 
your life, particularly analyzing and communi- 
cation, and apply them to your job. We have a 
different format, but it should not be a 
problem. Put your judgments up front, and sup- 
port them with clearly written, informative 
paragraphs. Some people can write better than 
others, but virtually all the people we hire 
should be able to write." 

I have watched enough talented people stumble over 
intelligence writing to wonder whether our attitude 
towards it erodes their confidence and impedes their 
development. Why do analysts with advanced 
degrees and presumably much experience in writing 
sometimes fall apart when trying to put together a 
paragraph for an intelligence publication? How many 
cases of writer's block have we noted? 

I suggest that as a test we modify our introduction to 
the intelligence arts by delivering a simple message 
at the outset: it is not mystical, it is communications! 




Deadly Prose 

When it comes to intelligence writing, analysts seem 
to be getting the message that boring is best, fol- 
lowed closely by dry, and uninteresting. Somehow 
the notion that we are just one of many sources 
available to consumers and that if we do not grab 
their attention we will lose them does not appear to 
be a prevailing view. The way our product is printed 
is first rate, our graphics outstanding, and even our 
formats are quite good. I am concerned that once a 
consumer starts to read, he or she may be turned off 
by soporific prose. 

Far too often, scholarly equates to indigestible, and 
our titles could cure insomnia. I believe that many a 
DI analyst writing about the beginning of World War 
III could title his or her paper, "Massive Exchange 
of Thermonuclear Weapons Likely to Cause Severe 
Economic Disruption and Tax Most Medical 
Facilities." This is a straightforward, clear, and 
accurate title— and deadly dull. 

I do not advocate using silly or irrelevant writing in 
our publications, -but it seems reasonable to try and 
use our prose to engage policymakers. If I had to sell 
my paper, would anyone want to buy it? 

Coordination 

Most analysts appear to treat coordination with 
almost the same degree of enthusiasm as a trip to the 
dentist. One does it because it is required and not 
out of any expectation that the product might be 
improved. The implicit goal of the analyst seemingly 
is to take an article completely through the process 
without it being touched by other human hands. 

Pride of authorship is just one factor in the distaste 
for coordination. We may also be recreating the 
norms of academic behavior. Professors and graduate 
students present their ideas and then defend them to 
the death. Although everyone is polite, it has never 
been clear to me whether anyone listens to anyone 
else — other than junior scholars who sit at the feet of 
the masters. On occasion, in convocations of aca- 
demics, I have felt like I was attending a Borgia 
family reunion. 
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Intelligence officers do not exist solely to impress 
anyone with their scholarship or to prove the fallacy 
of an opponent's views. Our responsibility is to 
provide policymakers with the kind of information 
and insight that will best enable them to do their 
jobs. If that comes from an individual effort, so be 
it. But in my mind there should be equal credit for 
the analyst who recognizes the clarity and relevance 
of another's ideas and incorporates them in his or her 
product. Thus, coordination should be an opportunity 
to gain additional knowledge or to test the validity 
of one's thoughts, not an exercise in stonewalling or 
one-upsmanship. 

Assuming that being fair-minded and receptive are 
the starting points for effective coordination, there 
are a number of'ways to improve the chances of 
getting a good result: 

• Be clear in your mind why you have chosen your 
particular analytical approach. Explore the alterna- 
tive explanations and also be prepared to depict 
your situation. 

• Treat every suggestion seriously. "That is the ~ 
stupidest idea I have ever heard" is not a 
response calculated to improve the coordination 
process. 



Take seriously the comment, "I do not understand 
what are you saying." We are in the communica- 
tions business. If someone who is familiar with a 
subject is confused, consider the impact on the 
poor policymaker. 

If you disagree with specific coordination com- 
ments, write alternative words before consulting 
directly with the coordinator. With words in hand, 
you are in a much better position to influence the 
course of discussion. 

Be willing to concede several minor points to gain 
a major one. It is hard to resist, "I gave you what 
you wanted on these three ideas, can't you move 
closer on this other one for me?" 



Still a Good Show 

In presenting these pitfalls and problems, I would be 
remiss if I did not put them in perspective. I believe 
that we in the DI and the Intelligence Community as 
a whole generally have the right to be proud about 
the work we do and the printed and oral intelligence 
that we provide. What I have pointed to as needing 
attention are deviations from a product norm that is 
of high quality, thoughtful and relevant. 



Be prepared to give ground. Ninety percent, or 
even 75 percent, of something is better than 100 
percent of nothing. 

Be willing to change words as an act of good 
faith. Words often offend when ideas do not. You 
may be able to keep the same analytical line, if 
you define it differently. 



It has been a matter of pride to me, however, that in- 
telligence organizations never rest on their laurels 
and are constantly be looking for ways to improve. If 
this were not so, it would have been pointless to 
write this article. 

This article is classified CONpTB^NTIAL (b)(3)(n) 



Be willing to be persuaded. Every old-timer can 
point to an occasion when he or she turned a 
piece 1 80 degrees on the basis of someone else's 
more perceptive argument. It does not happen 
often, but it is not a crime when it does. 
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